1. Field of the Invention
The present invention is related to a method of generating and outputting image information. In particular, the present invention is related to a system and a method of generating video image information, wherein a three dimensional image of above ground landscape including urban areas using aerial photographs taken from the sky by an aircraft or photographs taken above ground at street level is generated, and the present invention is also related to a system and a method of generating image information wherein a video picture of three dimensional city landscapes using photographs taken from various angles above ground is generated.
2. Description of the Related Art
In the past, as a method of three dimensional video picture of city landscapes it is known to obtain three dimensional images by consecutive display in real time by producing a three dimensional model by measuring three dimensional data of city buildings using a laser profiler, applying texture of wall images of buildings, and generating images of arbitrary perspective routes and arbitrary perspective directions along these perspective routes using a three dimensional image process. Another method of obtaining three dimensional data of city buildings uses images taken from different angles with regard to the same geographical ground point, and extracts data in a height direction using a stereo image process to generate three dimensional city models. In addition, there are also methods which do not always reproduce an actual match of a city but generate three dimensional image data by constructing three dimensional models of city space.
All of these methods require time and are costly for generating three dimensional structural elements, that is, three dimensional models, of a city. Furthermore, in order to make an exact match of the building exterior walls it is necessary to map exterior wall pattern data or image data (texture mapping) and this texture mapping data, in other words, data from all directions of a three dimensional object is essential for texture mapping, and obtaining this data and texture mapping requires considerable effort and cost. Thus, although the display of three dimensional images of arbitrary city section landscape from an arbitrary aerial route and line of vision is strongly desired, the accuracy of texture mapping can not be sufficiently realized, or the renewal cycle of data becomes longer, which produces shortcomings in the freshness and accuracy of displayed images. In addition, a method for indirectly producing three dimensional data is described in Japanese Laid Open Patent 2006-350852. That is, obtaining three dimensional data of an object using a common photographic object in two panoramic images as a corresponding point by employing a triangulation method. Producing what is called a central panoramic image in Japanese Laid Open Patent 2006-350852 involves calculating corresponding points for all points which requires three dimensional data in the panoramic image and measuring three dimensional data by a triangulation method. It is necessary to (1) measure corresponding points each time or (2) register corresponding points in advance. An image recognition process is required to automatically perform the method of (1) and it is extremely difficult to automatically calculate the point which becomes the peak of a polygon as a three dimensional coordinate. In addition, (2) is a method in which a process of calculating three dimensional coordinates in advance using a usual survey by photo is calculated each time a combination of the panoramic photographs which are used changes and is merely postponing the calculation, and load at the time of display becomes much larger compared to conventional CG technology. This is no different to conventional methods in which three dimensional data of a display object is calculated and texture is mapped and a fundamentally different technology to the present invention.
While as an image generation and browsing system using three dimensional simulated vision, Japanese Laid Open Patent 2002-185981 discloses a midair entertainment system via the internet using aerial photographs and includes features such as the ability to add building data, the method consistently uses [three dimensional object data which is applied and generated with a stereo process is performed on photographic data] or [three dimensional object data] of a building which is changed by a changing means, which is a fundamentally different technology to the present invention. While there is also a method which uses processing of land surface images taken by a satellite as in Ace Combat of ACECOMBAT, in the end the purpose of this method is to bring about a sense of reality to games and does not always match actual three dimensional landscapes. In addition, city space three dimensional simulated vision which can be seen in Second Life in SECONDLIFE displays virtual cities in three dimensions which are not compatible with actual maps. While there is meet-me using COCOR of MMET-ME, which is a three dimensional image generation and entertainment system of virtual city spaces compatible with actual maps for displaying city spaces more realistically compared to Second Life, it does not match reality in terms of exterior wall landscape and detailed structure of buildings. In addition, the technology of the present invention accumulates a large amount of multi-view point and multi-directional aerial photographs taken in advance, and the conventional technology for obtaining aerial photographs uses a digital format one dimensional line sensor from an analog camera type aerial camera owing to recent rapid process in digital technology or an aerial photographic camera using two dimensional video elements.
Furthermore, it is also being applied in the aerial photographic field by utilizing the high capabilities of a general purpose digital camera. A system for creating Ortho images from oblique photographs taken using a digital camera and dedicated software is proposed in Japanese Laid Open Patent 2002-357419. In addition, while the coordinates of a geographical feature are calculated by setting ground reference points in advance and calculating the camera position and orientation by referring to these reference points in an aerial photograph of a desired survey, it is possible to equip an aircraft with GPS (Global Positioning System) and IMU (Inertial Measurement Unit) and measure the camera position and orientation at the time of taking the image (Japanese Laid Open Patent 2004-245741). A technology for photographing a survey for aerial photograph using three CCD line sensors each with different photographying directions is proposed in Japan Registered Patent No. 2807622.